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INTRODUCTION 




Education "innovations" tend to 
come and go, usually without significant 
effects Enthusiastic proponents emphasize an 
innovation's presumed theoretical and 
practical benefits, urging immediate wide- 
spread adoption, opponents stress presumed 
shortcomings, urging restraint and caution But 
most educators simply shake their heads 
skeptically, observing, "This too shall pass " 
They are usually correct Unless preventive 
steps are taken, the outcome of the nation's 
current interest in incentives for superior 
teachers will probably be similar to the 
outcome of previous interest in the \ "pic much 
talk, some action, little follow-through and no 
lasting impact 




Incentives for teachers have once 
again been placed on America's agenda for 
improving education A Nation at Risk, the 
report of the National Commission on 
Excellence in Education, called for making 
teachers' salaries "professionally competitive, 
market-sensitive and performance-based ' In 
its report, Action for Excellence, the 
Education Commission of the States' Task 
Force on Education for Economic Growth 
urged state and local education agencies to 
improve recruiting, training and paying 
teachers The report called for "financial 
incentives" and "extraordinary rewards for 
extraordinary teachers expanded pay and 
recognition not just for reaching the upper 
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levels of seniority, but for reaching the upper 
levels of competence and effectiveness as 
well " The Twentieth Century Fund Task Force 
on Federal Elementary and Secondary 
Education Policy recommended that the 
federal government set up a master teacher 
incentive program to provide recognition and 
substantial grants to teachers of exceptional 
m<^rit President Reagan has emohasized 
incentives for excellence in his calls for school 
improvement, noting, "Secretary Bell and I 
have been pushing hard for a national agenda 
for excellence in education, and one of the first 
Items on it is the concept of merit pay for 
teachers If we want to achieve excellence , we 
must reward it It is a simple American 
philosophy that dominates many other 



professions, so why not this one*^" And the U S 
Congress, specifically the House Task Force 
on Merit Pay. tentatively endorsed the 
President's stand "despite mixed and 
inconclusive past results with performance- 
based pay in the private sector and in 
education, we support and encourage [state 
and local] experiments with performance- 
based pay " 

Yet. as the House Task Force report 
has recognized, performance-based incentive 
systems have so far not lived up to expecta- 
tions In a 1979 survey on merit pay. the 
Educational Research Service pointed out that 
differential pay plans for teachers have been 
tried since the early 1900s. with one wave of 
interest peaking in the 1920s and another in 
the mid-1 960s Only 4% of the nation's school 
districts were using any form of "merit pay" by 
1978, twice as many districts report they had 
abandoned merit pay systems 

Why , then, the few optimism of 

1984'? 

One reason for optimism is that 
recent research-based advances in knowledge 
about effective teaching and teacher assess- 
ment have encouraged people to conclude 
that problems plaguing past efforts can be 
solved The present political climate, which 
promotes free-enterprise solutions to social 
problems, supports the notion that offering 
incentives to teachers will increase education 
excellence Even the teacher unions, histori- 
cally hosiile to merit pay. have recently 
expressed a willingness to consider plans that 
offer differential rewards for differential 
performance, ptovided that teachers partici- 
pate voluntarily and that teachers help set 
performance criteria and assess performance 



But can the opportunity for new 
optimism be seized'^ And. if it is, will we be 
able 10 learn from the results'^ Research has 
demonstrated that well-performed evaluations 
of an innovative education practice can 
significantly affect the "no impact" cycle, first 
by helping to refine and improve the practice, 
and then by demonstrating its benefits to the 
wider audience that typically resists major 
changes Proper program evaluation is thus a 
useful step to take in the experiments with 
performance-based incentives now under way 
or being contemplated around the country 

Haphazard evaluations, though, 
may do more harmthan good Subjective and 
anecdotal reports, communicated in imprecise 
language, will do little to advance knowledge 
about the irriplementation of workable 
performance-based incentive systems or t*^eir 
benefits and costs Moreover, studies thai 
provide only the illusion of knowledge may 
encourage policy makers to draw erroneous 
conclusions about performance-based 
teacher incentives Not supnsingly. Hatry and 
Greiner were recently forced to conclude that 
there is "little convincing evidence (that is. 
evidence with even minimal rigor) — one way 
or another — on whethe. teacher merit 
pay plans have substantially affected 
student achievement, teacher retention rates 
or the ability to attract new (higher quality) 
teachers " The few evaluations undertaken to 
date have been almost totally subjective, and 
determining what types of "merit pay" systems 
were actually evaluated is difficult 

Unless this problem is eliminated 
in the ciJirent round of incentive experiments, 
the result is likely to be what it has been in the 
past inability to distinguish between the 
outcomes of performance-based systems and 
the outcom'^s of other kinds of incentive 
systems Five years hence, we may still be 
unable to answer the question. "Can superior 
teachers be rewarded'^" 
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This paper is intended to help avoid 
such an unfortunate outcome It provides a 
framework for 1. identifying performance- 
based incentive systems. 2. examining their 
underlying assumptions. 3. classifying 
program operations and 4. checking that a 
program as implemented is ready for 
evaluation 
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1. ©ENTIFYING PERFORMANCE-BASED INCENTI^ 




1 



The possibihty of learning from new 
expenmentswith teacher incentives is clouded 
by the confusing terminology educators and 
policy makers are using For the purposes of 
this paper, an incentive is any commonly 
valued reward whose delivery is contingent on 
tne attainment of some agreed-upon goal or 
standard * Incentives and rewards can 
therefore be quite various The anticipation of 
money is a widely valued incentive, and cash 
ts a widely ottered reward Material objects, 
events and opportunities to engage in 
desirable activities can also constitute 
rewards A range of rewards that might be 
useful in teacher incentive programs will be 
discussed in Section 3 For convenience, cash 
rewards will not be differentiated from other 
types of rewards in th'S preliminary discussion 



Trom a theoretical perspective, it is often necessary 
to treat rewards and incentives as different concepts 
an incentive is the effect of anticipating a reward 
This paper uses these concepts 





Generally, the goal of an incentive 
system ts to promote and maintain a certain 
level of behavior An incentive based on 
performance is one in which the reward is 
contingent on some predetermined aspect of 
task or job behavior; usually, a high level of 
quality or quantity in performance must be 
achieved before the reward is delivered In 
contrast, tr?ditional rewards for teachers are 
not based directly on performance but on some 
nneasure of status or job responsibility that 
IS presumed to correlate with performance In 
thiscase.statusmight refertoeducation level 
or years spent teaching Job responsibility 
nnight referto serving asdepartment chairper- 
son But to be performance-based, a reward 
mustbe deliveredonly when job performance 
has met or exceeded sonne predetermined 
standard 

Consider two common types of 
education reward systems In one system, 
teachers are awarded higher salaries as a 
function of their education and their teaching 
experience In theother, teachers are awarded 
higher salaries or bonuses as a function of their 
teaching skills, measured by their demonstra- 
tion of "effective teaching" behavior In the first 



system, rewards are contingent on course-tak- 
ing and persistence in employment, in the 
second, they are contingent on teaching 
behavior Only in the second system is the 
quality of job performance directly relevant 
The evaluation of job performance is the key 
ingredient of performance-based incentive 
systems, it is an ingredient not essential to 
other kinds of teacher incentive systems 

It IS possible to have mixed 
systems Consider two master teacher 
programs One identifies master teachers by 
determining that their job performance 
exceeds some designated standard and pays 
them more than teachers whose performance 
does not exceed the standard The second 
identifies master teachers on the basis of job 
performance but rewards them only if they 
assume extra responsibilities such as training 
new teachers The first program is primarily 
performance-based The second is primarily 
responsibility-based (although superior 
performance makesteachers eligible for extra 
responsibility) 

.1) 



The remainder of this paper will 
focus only on performance-based incentives 
because it is primarily the validity of the 
argument linking incentives to performance 
that must be tested by future program 
evaluations This approach does not deny the 
importance of incentives based on status or 
responsibility, which have historically formed 
the basis for almost all teacher compensation 
and reward systems and will likely continue to 
doso Rather, it directs attention tothe unique 
problems of assessing systems that make 
evaluation of pe 'ormance a prerequisite for 
exceptional rewards System evaluators must 
first be able to answer the question "Is the 
system rewarding teachers for superior 
performance'^" before the> can answer 'Does 
rewarding superior teachers improve educa- 
tion'^" 



2. CLARIFYING ASSUMPTIONS 




v/ertain assumptions about the 
nature of teaching excellence apply to all 
performance-based incentive systems and 
have implications for their design Failure to 
make these assumptions explicit can lead to 
unclear or misstated objectives or to a 
mismatch of objectives and activities It may 
also prevent meaningful evaluation 

Assumptions in the following three 
areas are crucial 

■ DIstrfbutlon of teaching skill: How is 
teaching siciil assumed to be distributed? 
Can oniy a few teachers achieve 
excellence, or can all teachers? Does the 
system emphasize identification of a 
select few or improvement of all? 

■ Interdependence: Is excellent teaching 
considered primarily an individual 
activity or a team effort? Can individutii 
teachers produce excellence, or does 
excellence transcend the individual 
classroom? 

■ Learner outcomes: Do processes (what 
teachers do) constitute teaching excel- 
lence or are leamer outcomes the 
measure of excellence? Is excellent 
teaching separable from its results? If 
leamer outcomes are crucial to excel- 
lence, how much are they affected by 
variables like student ability, previous 
learning, home and family environment, 
school environment? 

Consider first thr distnbution of 
teaching skills One might envision three 
different incentive systems resulting from 
different assumptions 
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■ It IS assumed that only a few teachers are or 
can be outstanding The objective of the in- 
centive system would be to identify them and 
encourage them to remain in teaching 
Quotas would be restricted The identity of 
reward recipients might be protected to 
avoid negative responses by their peers 

■ It IS assumed that only a few teachers are 
outstanding, but that many more could be, 
given more training and support The objec- 
tive would be to identify outstanding teachers 
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and encourage them to serve as models of 
excellence Quotas would be loose, and re- 
ward recipients wouia be encouraged to in- 
teract widely with other teachers 

■ It IS assumed that most teachers are already 
capable of excellence, needing only proper 
motivation to perform at peak ability The ob- 
jective woald be to motivate better perfor- 
mance No quotas would be set Every 
teacher would have an opportunity to earn 
a reward, and teachers who attain desig- 



nated performance standards would be pub- 
licly recognized 

An examole of a mismatch of 
objectives and design would be attempting to 
improve virtually all teachers by rewarding a 
tiny fraction and keeping their identities secret 

In the area of interdependence, one 
might assume that excellence is largely a 
function of collective efforts, say of all the 
teachers in a school One woald t^en establish 
performance standards for srhools and reward 
schools that meet or exceed these standards 
If this assumption were combined with the 
assumption that all teachers can be outstand- 
ing with training and support, the objective of 
the incentive system would be to identify 
outstanding schools' teachers and encourage 
them to serve as models of excellence 

Assumptions about the role of 
"learner outcomes" would manifest themselves 
in decisions about whether to ir. elude 
measures of student learning m performance 
standards If outcorTies were assumed not to 
be suitable measures, the objecti/e of the 
incentive system would be to influence 
teaching practices or school organiza*'on 

All three assumptions vary 
independently of each other and combine to 
oroduce different incentive designs The 
configuration of assumptions that underlie an 
incentive system influences objectives and 
implementation strategies " ^ystem designers 
examine their assumptions carefully, they can 
select the most appropriate objectives and 
strategies This, m turn, can set the stage for 
meaningful evaluation 



3. PROGRAM OPERATIONS 




Another way that incentiv ^ 
systems differ is in how they are structurec^ 
the components of performance-based 
incentive systems were ciasstfied, evaluatorc 
could more easily compare the benefits and 
costs of different rystems This section surveys 
types of incentives, methods of reward 
delivery, timing and types of performance 
standards and measures These variables 
appeared in an earlier review of the literature 
on the use of incentives in the schools 
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Types of Incentives 




Monetary Incentives 

As a general principle, the most 
effective reward satisfies the most salient 
needs perceived by the potential recipient 
without producing significant negative side 
effects Because perceived needs and the 
sal>ence of those needs differ considerably 
from person to person, rewards often differ in 
their effectiveness Th.s explains the popularity 
of flexible rewards like cash, which can readily 
be converted into a wide range of need- 
satisfytng materials and events But cash may 
have undesirable side effects such as taxability 
and high potential for engendering negative 
feelings hkeenvy while squeezing meaningful 
amounts of cash out of the constrained 
budgets of many school districts may be 
difficult * The following var,ations on cash as 
incentives ail require some form of evpendlure 
or obligation but they may avoid some of the 
negative side effects of direct pa' ients to 
teachers 

■ Drawing accounts that award recipients 
can use for educational materials or 
activities 

■ Educational scholarships or forgivable 
loans that permit recipients to acquire 
further education 



•Hatry and Gremer estimate that meaninvjful bonuses 
for teachers must exceed i 1,000 per year 



■ Paid activities such as educational travel 

■ Deferred benefits, such as extra retire- 
ment benefits 

■ A "menu" of rewards from ,vhich 
recipients can choose 

Note, however, that rewards of 
additional instructional resources can widen 
the performance gap between superior and 
other teachers, so a district that believes all 
teachers can improve would not want to use 
such rewards And any type of reward can 
prompt resentment 

Nonmonetary Incentives 

One of the more interesting 
features of current teacher incentive pians is 
the expanded use of nonmonetary rewards 
that do not require direct cash expenditures 
Money may in some instances be less cost 
effective than other rewards Under certain 
conditions, praise and knowledge of results 
can satisfy strongly perceived individual 
needs Many nonmonetary incentives have a 
long history of use in education Following are 
some nonmonetary rewards for mentonous 
eaching 

■ Public recognition: teacher of the year" 
awards 

■ Peer recognitior: designation of "master 
teachers" as successful professionals 

» Restructuring of the work environment: 
giving recipients more released time, 
options to work part time, more discretion 
in determining where and what to teach 

■ Recognition and awards for schools: the 
Secretary of Education's program of 
recognition for ef active schools 




Mil of these incentives are based 
primanly on praise and recognition To some 
extent, they are also based on feedback, 
because contingent praise conveys the 
informdtion that standards have been 
exceeded Some researchers have pointed out 
that knowledge of results can be rewarding in 
and of itself, if it conveys specific information 
about performance in a situation where 
successful performance is valued This means 
ihat any formal teacher evaluation system 
could be rewarding if it set measurable 
performance goals and provided feedback to 
teachers about how well they meet those goals 

Punishment 

The systematic connection of 
negative sanctions with poor performance is 
the other side of the incentiveG com Creating 
such a connection is an often-stated but 
seldom-achieved goal of many teacher 
evaluation systems Performance-based 
layoffs have occurred, however Johnson 
describes general instances and points out 
that tenure laws and collective bargaining 
agreements do not raise insurmountable 
barriers to the development of functional 
negative incentive" systems 

Methods of Delivering Rewards 
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Quotas 

In-'^^'^ntive Lystems may or may not 
restrict the number oi teacners who can earn 
awards If ti,e designer? of a system a'^sume 
that ^ " ^ -normally distributed (i e . 

along a beli-sh.'^O'j j curve), they may 
determine that seme arbitrary percentage 
(usually small) of teachers should be rewarded 
Designers who assume that all or most 



teachers are capable of excellence — since 
training, certification and selection require- 
ments have screened out teachers incapable 
of excellence — nnay avoid quotas 

Rewards for Performance 

Mechanisms for deltvennn 
rewards (usually nnonetary rewaids) to 
individuals include 

■ One-time cash bonuses, stipends, 
honoraria or supplemental contracts 

■ Permanent "merit" Increases on uniform 
salary scales (I.e., Increases granted 
above and beyond contractually guaran- 
teed cost-oMlvlngor longevity Increases) 

■ Use of a separate "merit" salary scale In 
addition to the uniform scale 

■ A "career ladder" that provides several 
pay scale ranges — for probationary, 
professional and master teachers, for 
example 

One variation on a basic bonus 
mechanism is a token system, su^h ac the 
"point' system developed m the Houston 
Independent School District's 'Second Mile 
Plan ' Under this syslenn, many different 
aspects of a teacher's performance and other 
qualifications are evaluated separateiv and 
translated into points, differential bonusesthat 
add as much as 30% to base pay depend on 
total points accumulated 

Rewards for Groups 

Systems for delivering rewards to 
groups either 1. provide each member of a 
designated group with the same reward or 2. 
allow ^up members to aetermine how their 
r vard wi). je apportioned The most common 
type of group is the entire staff of a singie 
school, as in Houston and Dallas Other groups 
are teams or departments within schools, 
intradistrict clusters of schools or even 
intrastate clusters Administrators and 
noncertificated personnel may be included in 
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The choice of delivery mechanism 
has implicattor s for the timing of awards 
Salary increments are iOrmally delivered 
annua'ly as part of the regularpersonnei review 
process Bonuses can be awarded annually or 
more often, depending on how frequently 
performance ts measured The deleterious 
effects of delay between measuring perfor- 
mance and reward can be mitigated by 
ensuring that feedback on performances is 
delivered rapidly 

A special delivery mechanism 
must usually be developed for nonmonetary 
rewards Although these can be less complex 
administratively than mechanisms for 
d'^livering cash, ceremonies must be 
organized, plaques engraved publicity 
arranged, and so forth The sooner a reward 
(or at least notification of an award) is 
delivered, the greater the incentive effect is 
likely to be 




incentive clusters if 't is assur^ed that 
nonteaching personnel enhance teaching 
effectiveness 

Timing and Reliability 

Delivering rewards soon after ihe 
qualifying evaluation ts very important If a 
teacher doubts the reliability of the delivery 
system or if rewards are delivered long after 
performance \t measured, incentive effects 
can be seriously weakened Potential 
recipients of rewards must understand Ihe 
delivery mechanism welt before they are 
expected to demonstrate meritorious perfor- 
mance The mechanism should operate 
according to plan 




Measures of Performance 



Valid and reliable measurements of 
performance are critical to any teacher 
incentive system While selecting appropriate 
measures is extremely difficult performance 
measurement techniques are readily 
catego^zed The majo^ decision for system 
designers is whether to use process measures 
or outcome measures In either case, the 
performance of individuals o. groups can be 
compared to the achievements of their peers 
or they can be compa''ed to ore-established 
standards 

Process Measures 

Following are some common 
measurement techniques that look at what 
teachers do 

■ Overall ratings (usually unstructured) by 
administrators 

■ Direct observation by peers, adminis- 
trators or trained outside raters using 
structured protocols 

■ Ratings by students or parents 

a Job performance tests or simulations 

■ Routine personnel records on absen- 
teeism, Inservice participation, otc. 

Outcome Measures 

Outcome . measures are based on 
such measurements of student learning as 
standardized achievement tests and criterion- 
referenced tests Stat'stical techniques can be 
used to reduce the influerice of environmental 
factorsor prior learning on outcome measuies 
Other measures include drop-out rates, 
absenteeism or saniplcs of student work 



4. IMPLEMENTATION ANALYSIS 





Ml ^ # 



a 



Before evaluating any program 
evaluators should be sure that the program hcS 
clear and acceptably slated goals and 
implementation activities thai are likely to 
accomplish them In addition, it must be 
possible to obtain from the program appro- 
priate information about operations and 
outcomes Through a process called "evalua- 
bility assessment," evaluators are urged to 
examine whether the program nneels the 
following requirerrier' 



a Stiled program goah' and actual goals 
are reasonably congrv ent. 

■ The hypothesized links «tween intended 
program inputs, activit s and outcomes 
are both logically and e ^plrically sound 
(I.e., are based on consi ^9nt reasoning 
end on the best available i ;dence about 
cause-effect relationship& 

■ Actual program Inputs and .tlvltie8(i.e., 
the program in operation) 'atch those 
that ware intended. 

This process, also cali I "im- 
plementation analysis," is a systemc c way of 
verifying that the orogram as imple nted is 
a performance-based incentive syste > c,nd is 
capable of being accurately evaluatec ^' or to 
initiating impact or cost/benefit studie 
Implementation analysis sets the stagt for 
successful evaluation of the new wave 
teacher incentive trials under way 
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Are Stated and 

Actual Goals Congruent? 

Four goals are most commonly 
advocated for new experiments with teacher 
incentives 

■ Attracting and retaining better quality 
ti'-ichers 

■ Improving teaching and learning 

■ Increasing accountability of teachers and 
schools to the public 

■ Increasing teachers' responsibilities In 
setting school direction 

Butmanytimesthestated goals of 
teacher incentive systems turn out, on closer 
examination, to differ from those the systems 
are actually designed to achieve Evaluations 

15 
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based on unclear or misstated goals wilt not 
be accurate More seriously, misunderstana- 
ings among participants m systems with 
misstated goals can easily jeopardize the 
success of the systems 

Assessment of an incentive system 
should begin by examining the congruenceof 
stated and actual goals Actual goals are 
inferred from the elements of job performance 
selected for measurement ?nd reward — the 
job standards that must be met or exceeded 
before incentives are awarded 'n other words, 
desired system outcomes are defined 
operationally by the performance measures 
and criteria of adequacy that have been 
adopted In performance-based systems 
criteria e^e most often applied to some 
measure of effective teaching An illustration of 
a teaching effectiveness performance 
measure is "maximizing engaged learning 
time", in a particular incentive system, the 
criterion of "using 75%of classtime in engaged 
learning" might be applied to this measure 
Teachers who consistently meet or exceed this 
designated standard are rewarded It may be 
inferred that the actual desired outcome of this 
system is "to improve teaching " 1 ne system s 
stated goal should reflect this 

Are the Hypothesized Links 
Between Inputs. Activities 
and Goals Logically and 
Empirically Sound? 



I he more direct and "tested" are 
the hypothesized links between the intended 
process components and outcomes of an 
incentive system, the more confident one can 
be that they are logically and empirically 
'jOund Figure 1 illustrates three simplified 
rationales linking performance measures and 
criteria to the goals for the performance-based 
system of which they are a part 

In the first example one might 
disagree with some of the hypothesized links 
but they are straightforward and each can be 
supported by empirical evidence available m 



the published research literature in the second 
example, the rationale can be similarly followed 
and IS at feast plausible as shown by the 
considerable popular support for this position, 
but empirical evidence that merit bonuses will 
attract a differeni (and better type of person) 
into teaching is lacking 

Now consider CAample 3, which 
illustrates an illogical rationale for a reward 
system designed to achieve the same goal of 
improving teaching as the first example In this 
case, the reward is given for a measure that is 
not stated m either the performance criterion 
or the system goal This system will not work 
because teachers are not rewarded for the 
behaviors on which they are evaluated If they 
see that they consistently meet or exceed the 
performance criterion without ever receiving a 
reward, they may become discouiaged and 
cease trying to meet it 




Similarly the reward system gives 
teachers no guidance about what behaviors it 
expects of them they do not know whether to 
increase their engaged learning time (and by 
how much) or to improve their written lesson 
plans or whether some yet undiscovered 
performance criterion is what really counts In 
any event, one cannot telhfthe system's goal 
has been achieved because it is not apparent 
what that goal really 's 

In implementation analysis, 
carefully mapping out the rationale linking a 
system's intended processes and outcomes 
can point out inconsistencies, gaps and 
probable failure points (In fact, this exercise 
should be done prior to implementation to find 
and correct weaknesses and inconsistencies 
in planning ) Implementation and evaluation 
should be deferred for any incentive system 
for which a logically straightforward and 
empirically defensible rationale cannot be 
articulated 



Do Actual Program Components 
Match Intents? 

Implementation analysis requires 
one further step examining the program as it 
existsin practice Adequately carrying out this 
step requires obser ationsand interviewswith 
thoGe who are imp'ementing the incentive 
system and are being affected by it The 
following issues must be addressed 




Figure 1. 

Three Types of Rationales for Performance-Based 
Reward Systems 



Performance Reward/ Behavior Motivated 

Criterion Incentive by Incentive Goal 



Examples Straightforward Rationale for Performance-Based Incentive System the performance 
criterion on which reward ~ ire based is a measure of effective teaching supported by research 



Teacher spends 
75% of class time in 
engaged learning 
activities 




Successfu' teach- 
ers recei ve cash 
bonuses 




All teachirs increase 
percentage of class 

time spent in 
engaged learn, ng 




Teaching is 
improved 


Example 2. Tenuous Rationale the performance cr'terion is not a direct measure of the goal 
there is little evidence suggesting that ability to receive merit awafos is a significant factor in 
career choice 


Teacher soends 
75% of class time in 
engaged learning 
activities 




Successful teach- 
ers receivecash 
bonuses 




Capableyoung 
people see that ex- 
cellent teachers are 
rewarded 




More capable 
young people 
choose tobe- 
come teachers 


Example 3. IHogica! Rationale the reward is based on dr srent factors than teachers evaluations 
It IS not clear what behavior is expected of the teachers nor what the goal of the system is 


Amount of class 
timeteachers 
spend in engaged 
learning ac tivities 




Teachers whose 
lesson plans are 
neat and detailed 
receive cash 
bonuses 




• 




Teaching is 
improved 



■ Extent to which implementers and par- 
ticipants understand and share the sanme 
intended outcomes 

An example of failure to achieve common 
understanding of goals, standards and 
procedures would occur wtiere teachers 
were not involved In planning the system 
and held dlff^^rent views from the adminis- 
tration and each other. 



■ Extent to which program activities are 
being implemented as planned 

For example, failure to train those who 
evaluate teachers could mean that they 
do not carrv out their responsibilities as 
accuratelv as pl<^nned. 



■ Extent to which unanticipated activities 
and negative side-effects Interfere with 
the validity of hypothesized cause-effect 
linkages 

A sudden budget shortfall In the middle 
of a multlyear experiment could eliminate 
money for rewards, or a new superinten- 
dent could reduce commitment to the 
program. Unrewarded teachers might be 
unexpectedly resentful rather than moti- 
vated. 

■ Likelihood that the program as Im- 
plemented can accomplish Its Intended 
outcomes. 

No implementation goes exactly as pian- 
neo; how much deviation will still permit 
goal achievement needs to be examined. 

If implementation analysis shows 
that the reality of the program does not match 
Its intent, evaluation should be deferred or 
redirected An important benefit of this stage 
of analysis is developing a set of recommenda- 
tions for imoroving implementation, also known 
as formative evaluation This potential argues 
for prompt and continuous monitoring of 
program operations 
O 



5. PROSPECTS FOR LEARNING FROM INCENTIVE EXPERBVIENTS 




Many new teacher incentive 
systems are currently operating or will begin 
operating in the next year Some systems will 
have state-mandated specifications (e g , 
systems in Florida, Virginia and Tennessee) 
Most nmII be locally developed in response to 
locally perceived needs and constraints Some 
will feature performarce-based incentives, 
others will combine performance-based and 
responsibility- or status-based incentives 
There will be many different combinations of 
assumptions about teaching excellence, goals 
and process components — each combination 
shaped by a different planning history and 
context What are the chances that we wiil learn 
enough this time around to decide whether 
superior teachers can indeed be rewarded'^ 
What will we be ab'e to learn about the benefits 
and costs of various reward systems'^ 




ERLC 




I he past suggests cautions For 
example, the nation was almost 15 years into 
an experiment with compensatory education 
before evaluation models and technical 
assistance centers were introduced that 
allowed generalization about the effects of the 
Elementary and Secondary Education Act 
(ESEA). Title I (now the Education Consolida- 
tion Improvement Act. Chapter 1) Without 
mandates, technical assistance, standard 
assessment models, some common terminol- 
ogy and quality assurance evaluations of 
teacher incentive systems may fare worse 



Use of a Common 
Conceptual Framework 

If evaluators use a common 
terminology like that proposed m this paper, 
they will provide a sound basis for discussing 
what has been attempted and accomplished 



Below are questions that evaluators can use to 
organize their documentation 

1 What IS the orimary factor upon which 
rewards depend — job performance or job 
responsibilities'^ If thie answer is joo re- 
sponsibilities, the system is not prima.-'ily 
performance-based, documentation 
should makethisclear 

2 What basic assumptions about teaching 

e xellenceundi-rlietf , incentive systern'^ 
a Distribution of teaching skill 
b Pole of interdependence 
c Inclusion of learner outcome^ 

o What components are intended'^ 
a Typos of reward 

(i) Monetary 

(ii) Nonmonetary 
(ill) Punitive 

b Who is rewarded'^ 

(i) Restrictive/nonrestricti\/e 
quotas 



(ii) Individuals/groups 
c Performance measurements 

(i) Process/outcome 

(ii) Norm-referenced/criterion- 
referenced 

Verification of System Design 
and Implementation 

Even given a common terminology, 
meaningful aggregation of information abouta 
system will be precluded if stated and de facto 
goals are not congruent, if system desig.. is 
not sound and if implementation does not 
maich intent Evaluators should be able to 
answer le following questions 

1 What are the stated goals of this mcenti 
system'? 

a Influencing recruitment 

b Inlluencing retention 

c Influencing motivation 

d Influencing studentachiovemen* 



2 What actual goaiscan bemferrea from its 
design'^ Are they congruent with stated 
goals'? 

3 is the stated rafonale for this system ex- 
QWCiP Is It both logically and empirically 
sound'? 

4 Does system implementation match stated 
tntenf? 



After acceptable models have 
been developed, technical assistance can be 
provided to stimulate use of the models and to 
support their correct use Publications 
Conferences, regional workshops and 
assistance at key sites can be useful 



If these additional steps are taken, 
new teacher incentive systems should be 
better designed, and we should be able to 
determine more precisely than ever before 
whether superior teachers can be rewarded 
and what the benefits of doing so are 



Additional Steps 

Implementation analysis estab- 
lishes that programs are in a condition to be 
evaluated The next step, of course, is actually 
*o evaluate costs, benefits and effeciiveness 
and to compare the results of different 
incentive systems Experience with ESEA Title 
I suggests some useful additional steps to 
prepare for evaluating performance-based 
teacher incentive systems Initially, evaluation 
models need to be developed for assessing 
both processes and outcomes and covering 
the range of likely goals and implementation 
components Models must be appropnate both 
for Simple incentive systems with one or two 
components and for complex systems with 
many components, including some not based 
on p.^rformance The models must allow 
evaiuators to produce information that is useful 
to local decision makers — to the school 
administrators, teachers, parents and other 
ciuzens of the comrriunity who will decide 
whether to keep, revise or drop performance- 
based incentives The models must accommo- 
date outcomes that are not readily quantifiable, 
dealing as they do in human values and 
attitudes If the models meet these needs, \t is 
likely they will be widely used, even if no state 
or federal funding is provided as encourage- 



ment 
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